System and method for verification of user identification based on multimedia content elements

ABSTRACT

A method and system for verifying an identity of a user accessing a user device are provided. The method includes receiving a request to verify the user identity accessing the user device; receiving a plurality of multimedia identification signals from the user device; querying a deep-content-classification (DCC) system to find a match between at least one concept structure associated with the user and the plurality of received multimedia identification signals, wherein the concept structure is created during a sign-in procedure; and sending an authentication notification upon determination of a match between the at least one concept structure and the plurality of received multimedia identification signals.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part (CIP) application of U.S.patent application Ser. No. 13/602,858 filed Sep. 4, 2012, now U.S. Pat.No. 8,868,619 which is a continuation of U.S. patent application Ser.No. 12/603,123, filed on Oct. 21, 2009, now issued as U.S. Pat. No.8,266,185, which is a continuation-in-part of:

(1) U.S. patent application Ser. No. 12/084,150 having a filing date ofApr. 7, 2009, now granted as U.S. Pat. No. 8,655,801, which is theNational Stage of International Application No. PCT/IL2006/001235, filedon Oct. 26, 2006, which claims foreign priority from Israeli ApplicationNo. 171577 filed on Oct. 26, 2005 and Israeli Application No. 173409filed on 29 Jan. 2006;

(2) U.S. patent application Ser. No. 12/195,863, filed Aug. 21, 2008,now issued as U.S. Pat. No. 8,326,775, which claims priority under 35USC 119 from Israeli Application No. 185414, filed on Aug. 21, 2007, andwhich is also a continuation-in-part of the above-referenced U.S. patentapplication Ser. No. 12/084,150, now U.S. Pat. No. 8,655,801;

(3) U.S. patent application Ser. No. 12/348,888, filed Jan. 5, 2009, nowpending, which is a CIP of U.S. patent application Ser. No. 12/084,150,now U.S. Pat. No. 8,655,801, having a filing date of Apr. 7, 2009 andU.S. patent application Ser. No. 12/195,863 filed on Aug. 21, 2008, nowU.S. Pat. No. 8,326,775; and

(4) U.S. patent application Ser. No. 12/538,495, filed Aug. 10, 2009,now issued as U.S. Pat. No. 8,312,031, which is a CIP of U.S. patentapplication Ser. No. 12/084,150 having a filing date of Apr. 7, 2009,now U.S. Pat. No. 8,655,801, U.S. patent application Ser. No.12/195,863, filed on Aug. 21, 2008, now U.S. Pat. No. 8,326,775; andU.S. patent application Ser. No. 12/348,888, filed Jan. 5, 2009. All ofthe applications referenced above are herein incorporated by reference.

TECHNICAL FIELD

The present invention relates generally to the analysis of multimediacontent elements, and more specifically to a system for verifying useridentification based on an analysis of multimedia content elements.

BACKGROUND

The last decade has seen rapid growth in the number of mobile devicesand their uses. Mobile devices include, but are not limited to, cellularphones, smart phones, tablet computers, and the like. However, as aresult of their popularity and functionality, mobile devices have becomea burgeoning target for theft. In addition, mobile devices are typicallyused to store private and/or confidential information. Such mobiledevices are typically installed with applications (e.g. emails, etc.)that provide an access to or contain private and/or confidentialinformation. Consequently, mobile devices have become more sensitive andvaluable. Thus, security measures are usually required to verify theidentity of the user before allowing access to the mobile devices.

Several applications for securing the access to the mobile devices arecurrently available. Unfortunately, such applications are static bynature and are usually configured to authenticate the useridentification based on only one, or in some cases two parameters, i.e.,fingerprint, voice recognition or eye scan, and so on. The use of apasscode alone has become a less reliable means to authenticate theuser. This is due to a computer hacker's ability to locate, copy, orelectronically identify or track the required password, usingspecialized software programs. Alternative prior art solutions allowverification of the user identity based on implicit inputs entered by auser of the device for the purpose of authentication, which are thenverified by the device's operating system. Such inputs include apassword, a passcode, or a fingerprint identifier. The passcode may bein a sequence of digits or a geometric shape drawn by the user on thedevice's display.

Nevertheless, the use of a password or passcode is a less secure meansto verify the user identity because a passcode can be tracked, locked,or discovered using malicious software programs, and further, some userssimply do not keep their password protected. The use of a fingerprinthas also not proven to provide a reliable method to verify a user, asfingerprints can be easily duplicated or simulated to hack to the mobiledevice.

Therefore, it would be advantageous to provide a solution that overcomesthe deficiencies for verifying the user identity for at least providingan access to at least the user's mobile device.

SUMMARY

Certain embodiments disclosed herein include a system and for method forverifying an identity of a user accessing a user device. The methodcomprises receiving a request to verify the user identity accessing theuser device; receiving a plurality of multimedia identification signalsfrom the user device; querying a deep-content-classification (DCC)system to find a match between at least one concept structure associatedwith the user and the plurality of received multimedia identificationsignals, wherein the concept structure is created during a sign-inprocedure; and sending an authentication notification upon determinationof a match between the at least one concept structure and the pluralityof received multimedia identification signals.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter disclosed herein is particularly pointed out anddistinctly claimed in the claims at the conclusion of the specification.The foregoing and other objects, features and advantages of theinvention will be apparent from the following detailed description takenin conjunction with the accompanying drawings.

FIG. 1 is a diagram of a DCC system for creating concept structures.

FIG. 2 is a flowchart illustrating the operation of the patch attentionprocessor of the DCC system.

FIG. 3 is a block diagram depicting the basic flow of information in alarge-scale video matching system.

FIG. 4 is a diagram showing the flow of patches generation, responsevector generation, and signature generation in a large-scalespeech-to-text system.

FIG. 5 is a flowchart illustrating the operation of the clusteringprocessor of the DCC system.

FIG. 6 is a flowchart illustrating the operation of the conceptgenerator of the DCC system.

FIG. 7 is a diagram of a network system utilized to describe certaindisclosed embodiments.

FIG. 8 is a flowchart illustrating a method for verifying useridentification based on multimedia identification parameters (MMIDPs)according to one embodiment.

FIG. 9 is a flowchart illustrating a method for verifying theidentification of the user according to an embodiment.

DETAILED DESCRIPTION

The embodiments disclosed herein are only examples of the many possibleadvantageous uses and implementations of the innovative teachingspresented herein. In general, statements made in the specification ofthe present application do not necessarily limit any of the variousclaimed inventions. Moreover, some statements may apply to someinventive features but not to others. In general, unless otherwiseindicated, singular elements may be in plural and vice versa with noloss of generality. In the drawings, like numerals refer to like partsthrough several views.

FIG. 1 shows an exemplary and non-limiting diagram of a DCC system 100for creating concept structures. The DCC system 100 is configured toreceive multimedia data elements (MMDEs), for example from the Internetvia the network interface 160. The MMDEs include, but are not limitedto, images, graphics, video streams, video clips, audio streams, audioclips, video frames, photographs, images of signals, combinationsthereof, and portions thereof. The images of signals are images such as,but not limited to, medical signals, geophysical signals, subsonicsignals, supersonic signals, electromagnetic signals, and infraredsignals.

The MMDEs may be stored in a database (DB) 150 or kept in the DB 150 forfuture retrieval of the respective multimedia data element. Such areference may be, but is not limited to, a universal resource locator(URL). Every MMDE in the DB 150, or referenced therefrom, is thenprocessed by a patch attention processor (PAP) 110 resulting in aplurality of patches that are of specific interest, or otherwise ofhigher interest than other patches. A more general pattern extraction,such as an attention processor (AP) may also be used in lieu of patches.The AP receives the MMDE that is partitioned into items; an item may bean extracted pattern or a patch, or any other applicable partitiondepending on the type of the MMDE. The functions of the PAP 110 aredescribed herein below in more detail.

Those patches that are of higher interest are then used by a signaturegenerator (SG) 120 to generate signatures respective of the patch. Theoperation of the signature generator (SG) 120 is described in moredetail herein below. A clustering process (CP) 130 initiates a processof inter-matching of the signatures once it determines that there are anumber of patches that are above a predefined threshold. The thresholdmay be defined to be large enough to enable proper and meaningfulclustering. With a plurality of clusters a process of clusteringreduction takes place so as to extract the most useful data about thecluster and keep it at an optimal size to produce meaningful results.The process of cluster reduction is continuous. When new signatures areprovided after the initial phase of the operation of the CP 130, the newsignatures may be immediately checked against the reduced clusters tosave on the operation of the CP 130. A more detailed description of theoperation of the CP 130 is provided herein below.

A concept generator (CG) 140 operates to create concept structures fromthe reduced clusters provided by the CP 130. Each concept structurecomprises a plurality of metadata associated with the reduced clusters.The result is a compact representation of a concept that can now beeasily compared against a MMDE to determine if the received MMDE matchesa concept structure stored, for example in the DB 150, by the CG 140.This can be done, for example and without limitation, by providing aquery to the DCC system 100 for finding a match between a conceptstructure and a MMDE. A more detailed description of the operation ofthe CG 140 is provided herein below.

It should be appreciated that the DCC system 100 can generate a numberof concept structures significantly smaller than the number of MMDEs.For example, if one billion (10⁹) MMDEs need to be checked for a matchagainst another one billion MMDEs, typically the result is that no lessthan 10⁹×10⁹=10¹⁸ matches have to take place, a daunting undertaking.The DCC system 100 would typically have around 10 million conceptstructures or less, and therefore at most only 2×10⁶×10⁹=2×10¹⁵comparisons need to take place, a mere 0.2% of the number of matchesthat have had to be made by other solutions. As the number of conceptstructures grows significantly slower than the number of MMDEs, theadvantages of the DCC system 100 would be apparent to one with ordinaryskill in the art.

The operation of the PAP 110 will now be provided in greater detail withrespect to an image as the MMDE. However, this should not be understoodas to limit the scope of the invention; other types of MMDEs arespecifically included herein and may be handled by the PAP 110.

FIG. 2 depicts an exemplary and non-limiting flowchart 200 of theoperation of the PAP 110. In S210 the PAP 110 receives a MMDE from asource for such MMDEs. Such a source may be a system that feeds the DCCsystem 100 with MMDEs, or other sources for MMDEs, for example theworld-wide-web (WWW). In S220 the PAP 110 creates a plurality of patchesfrom the MMDE. A patch of an image is defined by, for example, its size,scale, location and orientation. A patch may be, for example and withoutlimitation, a portion of an image of a size 20 pixels by 20 pixels of animage that is 1,000 pixels by 500 pixels. In the case of audio, a patchmay be a segment of audio 0.5 seconds in length from a 5 minute audioclip. In S230 a patch not previously checked is processed to determineits entropy. The entropy is a measure of the amount of interestinginformation that may be present in the patch. For example, a continuouscolor of the patch has little interest while sharp edges, corners orborders, will result in higher entropy representing a lot of interestinginformation. The plurality of statistically independent cores, theoperation of which is discussed in more detail herein below, is used todetermine the level-of-interest of the image and a process of votingtakes place to determine whether the patch is of interest or not.

In S240, it is checked whether the entropy was determined to be above apredefined threshold, and if so execution continues with S250;otherwise, execution continues with S260. In S250 the patch havingentropy above the threshold is stored for future use by the SG 120 in,for example, DB 150. In S260 it is checked whether there are morepatches of the MMDE to be checked, and if so execution continues withS220; otherwise execution continues with S270. In S270 it is checkedwhether there are additional MMDEs, and if so execution continues withS210; otherwise, execution terminates. It would be appreciated by thoseof skill in the art that this process reduces the information that mustbe handled by the DCC system 100 by focusing on areas of interest in theMMDEs rather than areas that are less meaningful for the formation of aconcept structure.

A high-level description of the process for large scale video matchingperformed by the Matching System is depicted in FIG. 3. Video contentsegments 2 from a Master DB 6 and a Target DB 1 are processed inparallel by a large number of independent computational Cores 3 thatconstitute the Architecture. Further details on the computational Coresgeneration are provided below. The independent Cores 3 generate adatabase of Robust Signatures and Signatures 4 for Targetcontent-segments 5 and a database of Robust Signatures and Signatures 7for Master content-segments 8. An exemplary and non-limiting process ofsignature generation for an audio component is shown in detail in FIG.4. Referring back to FIG. 3, at the final step, Target Robust Signaturesand/or Signatures are effectively matched, by a matching algorithm 9, toMaster Robust Signatures and/or Signatures database to find all matchesbetween the two databases.

A brief description of the operation of the SG 120 is thereforeprovided, this time with respect to a MMDE which is a sound clip.However, this should not be understood as to limit the scope of theinvention and other types of MMDEs are specifically included herein andmay be handled by SG 120. To demonstrate an example of signaturegeneration process, it is assumed, merely for the sake of simplicity andwithout limitation on the generality of the disclosed embodiments, thatthe signatures are based on a single frame, leading to certainsimplification of the computational core's generation. The MatchingSystem shown in FIG. 3 is extensible for signatures generation capturingthe dynamics in-between the frames and the information of the frame'spatches.

The signatures generation process will be described with reference toFIG. 4. The first step in the process of signatures generation from agiven speech-segment is to break-down the speech-segment to K patches 14of random length P and random position within the speech segment 12. Thebreak-down is performed by the patch generator component 21. The valueof K is determined based on optimization, considering the tradeoffbetween accuracy rate and the number of fast matches required in theflow process of the Matching System. In the next step, all the K patchesare injected in parallel to all L computational Cores 3 to generate Kresponse vectors 22. The vectors 22 are fed into the SG 120 to produce aSignatures and Robust Signatures 4.

In order to generate Robust Signatures, i.e., Signatures that are robustto additive noise L (where L is an integer equal to or greater than 1)computational cores are utilized in the Matching System. A frame i isinjected into all the Cores. The computational cores 3 generate twobinary response vectors: {right arrow over (S)} which is a Signaturevector, and {right arrow over (RS)} which is a Robust Signature vector.

For generation of signatures robust to additive noise, such asWhite-Gaussian-Noise, scratch, etc., but not robust to distortions, suchas crop, shift and rotation, etc., a core C_(i)={n_(i)} (1≦i≦L) mayconsist of a single leaky integrate-to-threshold unit (LTU) node or morenodes. The node n_(i) equations are:

$V_{i} = {\sum\limits_{j}{w_{ij}k_{j}}}$n_(i)=□(Vi−Th_(x)); □ is a Heaviside step function; w_(ij) is a couplingnode unit (CNU) between node i and image component j (for example,grayscale value of a certain pixel j); k_(j) is an image component j(for example, grayscale value of a certain pixel j); Th_(x) is aconstant Threshold value, where x is ‘S’ for Signature and ‘RS’ forRobust Signature; and C_(i), is a Coupling Node Value.

The Threshold values Th_(x) are set differently for Signature generationand for Robust Signature generation. For example, for a certaindistribution of V_(i), values (for the set of nodes), the thresholds forSignature (Th_(S)) and Robust Signature (Th_(RS)) are set apart, afteroptimization, according to at least one or more of the followingcriteria:For: V _(i) >Th _(RS)1−p(V>Th _(S))−1−(1−ε)^(l)<<1  Ii.e., given that I nodes (cores) constitute a Robust Signature of acertain image I, the probability that not all of these I nodes willbelong to the Signature of same, but noisy image, {tilde over ( )} issufficiently low (according to a system's specified accuracy).p(V _(i) >Th _(RS))≈l/L  IIi.e., approximately l out of the total L nodes can be found to generateRobust Signature according to the above definition.

III: Both Robust Signature and Signature are generated for certain framei.

It should be understood that the creation of a signature is aunidirectional compression where the characteristics of the compresseddata are maintained but the compressed data cannot be reconstructed.Therefore, a signature can be used for the purpose of comparison toanother signature without the need of comparison of the original data.The detailed description of the Signature generation can be found U.S.Pat. Nos. 8,326,775 and 8,312,031, assigned to common assignee, whichare hereby incorporated by reference for all the useful information theycontain.

Computational Core generation is a process of definition, selection andtuning of the Architecture parameters for a certain realization in aspecific system and application. The process is based on several designconsiderations, such as: (a) The Cores should be designed so as toobtain maximal independence, i.e., the projection from a signal spaceshould generate a maximal pair-wise distance between any two Cores'projections into a high-dimensional space; (b) The Cores should beoptimally designed for the type of signals, i.e., the Cores should bemaximally sensitive to the spatio-temporal structure of the injectedsignal, for example, and in particular, sensitive to local correlationsin time and space. Thus, in some cases a Core represents a dynamicsystem, such as in state space, phase space, edge of chaos, etc., whichis uniquely used herein to exploit their maximal computational power,and, (c) The Cores should be optimally designed with regard toinvariance to a set of signal distortions, of interest in relevantapplications. Detailed description of the Computational Core generation,the computational architecture, and the process for configuring suchcores is discussed in more detail in the co-pending U.S. patentapplication Ser. No. 12/084,150 referenced above.

Hence, signatures are generated by the SG 120 responsive of patchesreceived either from the PAP 110, or retrieved from the DB 150, asdiscussed hereinabove. It should be noted that other ways for generatingsignatures may also be used for the purpose the DCC system 100.Furthermore, as noted above, the array of computational cores may beused by the PAP 110 for the purpose of determining if a patch has anentropy level that is of interest for signature generation according tothe principles of the invention. The generated signatures are stored,for example, in the DB 150, with reference to the MMDE and the patch forwhich it was generated thereby enabling back annotation as may benecessary.

Portions of the CP 130 have been discussed in detail in the co-pendingU.S. patent application Ser. No. 12/507,489, entitled “UnsupervisedClustering of Multimedia Data Using a Large-Scale Matching System”,filed Jul. 22, 2009, assigned to common assignee (the “'489Application”), and which is hereby incorporated for all that itcontains. In accordance with an embodiment an inter-match process andclustering thereof is utilized. The process can be performed onsignatures provided by the SG 120. It should be noted though that thisinter-matching and clustering process is merely an example for theoperation of the CP 130 and other inter-matching and/or clusteringprocesses may be used for the purpose of the invention.

Following is a brief description of the inter-match and clusteringprocess. The unsupervised clustering process maps a certaincontent-universe onto a hierarchical structure of clusters. Thecontent-elements of the content-universe are mapped to signatures, whenapplicable. The signatures of all the content-elements are matched toeach other, and consequently generate the inter-match matrix. Thedescribed clustering process leads to a set of clusters. Each cluster isrepresented by a small/compressed number of signatures, for examplesignatures generated by SG 12 as further explained hereinabove, whichcan be increased by variants. This results in a highly compressedrepresentation of the content-universe. A connection graph between themultimedia data elements of a cluster may be stored. The graph can thenbe used to assist a user searching for data to move along the graph inthe search of a desired multimedia data element.

In another embodiment, upon determination of a cluster, a signature forthe whole cluster may be generated based on the signatures of themultimedia data elements that belong to the cluster. It should beappreciated that using a Bloom filter may be used to reach suchsignatures. Furthermore, as the signatures are correlated to someextent, the hash functions of the Bloom filter may be replaced bysimpler pattern detectors, with the Bloom filter being the upper limit.

While signatures are used here as the basic data elements, it should berealized that other data elements may be clustered using the techniquesdiscussed above. For example, if a system generating data items is used,the data items generated may be clustered according to the disclosedprinciples. Such data items may be, without limitation, multimedia dataelements. The clustering process may be performed by dedicated hardwareor by using a computing device having storage to store the data itemsgenerated by the system and then performing the process described hereinabove. Then, the clusters can be stored in memory for use as may bedeemed necessary.

The CP 130 further uses an engine designed to reduce the number ofsignatures used in a structure, in a sense, extracting only the mostmeaningful signatures that identify the cluster uniquely. This can bedone by testing a removal of a signature from a cluster and checking ifthe MMDEs associated with the cluster still are capable of beingrecognized by the cluster through signature matching.

The process of signature extraction is on-going as the DCC system 100operates. It should be noted that after initialization, upon signaturegeneration by the SG 120 of a MMDE, its respective signature is firstchecked against the clusters to see if there is a match. If a match isfound, it may not be necessary to add the signature to the cluster orclusters, but rather to simply associate the MMDE with the identifiedcluster or clusters. However, in some cases where additional refinementof the concept structure is possible, the signature may be added, or attimes it may even replace one or more of the existing signatures in thereduced cluster. If no match is found then the process of inter-matchingand clustering may take place.

FIG. 5 depicts an exemplary and non-limiting flowchart 500 of theoperation of the CP 130. In S510 a signature of a MMDE is received, forexample from the SG 120. In S520 it is checked whether the signaturematches one or more existing clusters and if so execution continues withS550; otherwise, execution continues with S530. In S530 an inter-matchbetween a plurality of signatures previously received by the DCC system100 is performed, for example in accordance with the principles of the'489 Application. As may be necessary the DB 150 may be used to storeresults or intermediate results as the case may be, however, othermemory elements may be used. In S540 a clustering process takes place,for example in accordance with the principles of the '489 Application.As may be necessary the DB 150 may be used to store results orintermediate results as the case may be, however, other memory elementsmay be used.

In S550, the signature identified to match one or more clusters isassociated with the existing cluster(s). In S560 it is checked whether aperiodic cluster reduction is to be performed, and if so executioncontinues with S570; otherwise, execution continues with S580. In S570the cluster reduction process is performed. Specifically, the purpose ofthe operation is to ensure that in the cluster there remains the minimalnumber of signatures that still identify all of the MMDEs that areassociated with the signature reduced cluster (SRC). This can beperformed, for example, by attempting to match the signatures of each ofthe MMDEs associated with the SRC having one or more signatures removedtherefrom. The process of cluster reduction for the purpose ofgenerating SRCs may be performed in parallel and independently of theprocess described herein above. In such a case after either S560 or S570the operation of S580 takes place. In S580 it is checked whether thereare additional signatures to be processed and if so execution continueswith S510; otherwise, execution terminates. SRCs may be stored inmemory, such as DB 150, for the purpose of being used by other elementscomprising the DCC system 100.

The CG 140 performs two tasks, it associates metadata to the SRCsprovided by the CP 130 and it associates between similar clusters basedon commonality of metadata. Exemplary and non-limiting methods forassociating metadata with MMDEs is described in U.S. patent applicationSer. No. 12/348,888, entitled “Methods for Identifying Relevant Metadatafor Multimedia Data of a Large-Scale Matching System”, filed on Jan. 5,2009, assigned to common assignee (the “'888 Application”), and which ishereby incorporated for all that it contains. One embodiment of the '888Application includes a method for identifying and associating metadatato input MMDEs. The method comprises comparing an input first MMDE to atleast a second MMDE; collecting metadata of at least the second MMDEwhen a match is found between the first MMDE and at least the secondMMDE; associating at least a subset of the collected metadata to thefirst MMDE; and storing the first MMDE and the associated metadata in astorage.

Another embodiment of the '888 Application includes a system forcollecting metadata for a first MMDE. The system comprises a pluralityof computational cores enabled to receive the first MMDE, each corehaving properties to be statistically independent of each other core,each generates responsive to the first MMDE a first signature elementand a second signature element, the first signature element being arobust signature; a storage unit for storing at least a second MMDE,metadata associated with the second MMDE, and at least one of a firstsignature and a second signature associated with the second MMDE, thefirst signature being a robust signature; and a comparison unit forcomparing signatures of MMDEs coupled to the plurality of computationalcores and further coupled to the storage unit for the purpose ofdetermining matches between multimedia data elements; wherein responsiveto receiving the first MMDE the plurality of computational coresgenerates a respective first signature of said first MMDE and/or asecond signature of said first MMDE, for the purpose of determining amatch with at least a second MMDE stored in the storage and associatingmetadata associated with the at least second MMDE with the first MMDE.

Similar processes to match metadata with a MMDE or signatures thereofmay be used. Accordingly, each SRC is associated with metadata which isthe combination of the metadata associated with each of the signaturesthat is included in the respective SRC, preferably without repetition ofmetadata. A plurality of SRCs having metadata may now be associated toeach other based on the metadata and/or partial match of signatures. Forexample, and without limitation, if the metadata of a first SRC and themetadata of a second SRC overlap more than a predetermined thresholdlevel, for example 50% of the metadata match, they may be consideredassociated clusters that form a concept structure. Similarly, a secondthreshold level can be used to determine if there is an associationbetween two SRCs where at least a number of signatures above the secondthreshold are identified as a match with another SRC. As a practicalexample one may want to consider the concept of Abraham Lincoln whereimages of the late President and features thereof, appear in a largevariety of photographs, drawings, paintings, sculptures and more and areassociated as a concept structure of the concept “Abraham Lincoln”. Eachconcept structure may be then stored in memory, for example, the DB 150for further use.

FIG. 6 shows an exemplary and non-limiting flowchart 600 of theoperation of the CG 140. In S610 the CG 140 receives a SRC from eitherthe CP 130 or by accessing memory, for example, the DB 150. In S620metadata are generated for the signatures of the SRC, for example inaccordance with the principles described hereinabove. A list of themetadata is created for the SRC preferably with no metadata duplication.In one embodiment the commonality of metadata is used to signify thestrength of the metadata with respect to a signature and/or the SRC,i.e., a higher number of metadata repetitions is of more importance tothe SRC than a lower number of repetitions. Furthermore, in oneembodiment a threshold may be used to remove those metadata that have asignificantly low rate of repetition as not being representative of theSRC.

In S630 the SRC is matched to previously generated SRCs to attempt tofind various matches, as described for example, hereinabove in moredetail. In S640, it is checked if at least one match was found and ifso, execution continues with S650; otherwise, execution continues withS660. In S650 the SRC is associated with one or more of the conceptstructures to which the SRC has been shown to match. In S660 it ischecked whether additional SRCs are to be received and if so executioncontinues with S610; otherwise, execution terminates.

A person skilled in the art would now appreciate the advantages of theDCC system 100 and methods thereof. The DCC system 100 is capable ofcreating automatically and in an unsupervised fashion concept structuresof a wide variety of MMDEs. When checking a new MMDE it may be checkedagainst the concept structures stored, for example, in the DB 150, andupon detection of a match provide the concept information about theMMDE. With the number of concept structures being significantly lowerthan the number of MMDEs the solution is cost effective and scalable forthe purpose of identification of content of a MMDE.

According to various disclosed embodiments, the system and method areprovided to verify an identity of a user accessing a user device. Thedisclosed system and method utilizes the DCC system 100 for theverification. The user identity is not authenticated or verified usingany passcode, password or fingerprints, but rather by using a pluralityof multimedia identification signals. In particular, a multimediaidentification signal is a multimedia data element (MMDE) which may be,for example, an image, a video signal, an audio signal, a combinationthereof, and so on. The multimedia identification signals may furtherinclude an audio gesture or a visual gesture as input by the user on theuser device. A video gesture may be, for example, the user's lipsdynamics or facial expressions, as well as surrounding parametersrelated to the user such as places, objects, and the like.

The multimedia identification signals are captured by or stored in theuser device. In one embodiment, the DCC system 100 generates a set ofsignatures for such multimedia signals and a concept structure based onthe set of signatures. When the user requests to access the device(e.g., unlock the device), signatures of the multimedia signals arere-generated and matched against the concept structure or previouslygenerated signatures. If the signatures are matched, then the user isauthenticated.

FIG. 7 shows an exemplary and non-limiting schematic diagram of anetwork system 700 utilized to describe the various disclosedembodiments. A network 710 is used as a means for communication betweendifferent elements of the network system 700. The network 710 may be theInternet, the world-wide-web (WWW), a local area network (LAN), a widearea network (WAN), a metro area network (MAN), and other networkscapable of enabling communication between the elements of the networksystem 700.

Further communicatively connected to the network 710 is a user device720. The user device 720 may be, for example, a personal computer (PC),a personal digital assistant (PDA), a mobile phone, a smart phone, atablet computer, and other kinds of wired and mobile appliances,equipped with browsing, viewing, listening, filtering, and managingcapabilities. In one embodiment, the user device 720 also includes oneor more multimedia capturing units (not shown), such as a camera, avideo camera, a microphone, a touch screen display, and the like. Thecapturing units of the device 720 are utilized to capture multimediasignals that serve as the multimedia identification signals. The userdevice 720 may also include a memory (not shown) utilized to maintainmultimedia signals previously stored by the user. For example, pictures,video clips, audio clips, etc. taken by the user and previously capturedby the user and stored in the user device's 720 memory.

Also connected to the network 710 is a server 730 configured to performthe process of verifying the user identity and configured to access thedevice 720. To this end, the server 730 is connected to a DCC system 740and a signature generator 750. The DCC system 740 is configured andoperates as the DCC system 100 discussed in detail above. The signaturegenerator 750 is configured and operates as the SG 120. In certainconfigurations, the SG of the DCC system is utilized as the signaturegenerator 750. The DCC system 740 and signature generator 750 may beconnected through the server 730 to the network 710 or through a directconnection. In certain configurations, the DCC system 740 and signaturegenerator 750 may be embedded in the server 730. It should be noted thatthe server 730 typically comprises a processing unit and a memory (notshown). The processor is coupled to the memory (not shown), which isconfigured to contain instructions that can be executed by theprocessing unit. The server 730 also includes a network interface (notshown) to the network 710.

The network system 700 also includes a database 760 which is configuredto store the multimedia identification signals and signatures thereofrequired for the user authentication. In one embodiment, the database ofthe DCC system may serve as the database 760.

According to the embodiments disclosed herein, in order to allow accessto the user device 720, a user of the device 720 is required to open anaccount in the server 730, for example, through a sign-in procedure.During this procedure a plurality of multimedia identification signalsare sent from the user device 720 to the server 730. The user of thedevice 720 may capture such signals using one or the sensors of thedevice 720. Alternatively or collectively, the user of the device 720may designate previous captured signals (e.g., pictures) stored in theuser device's 720 memory.

The sign-in procedure may be initiated by an agent (e.g., a script)executed on the device 720 which provides the multimedia identificationsignals and sends these signals to the server 730. In anotherembodiment, the sign-in procedure may be executed by the server 730which provides the instructions to the user on, e.g., a web-browser, oran application installed on the user device 720. In one embodiment, theadditional parameters are sent to the server 730, such as an IP addressof the computing device, time, date, a browser type, an operating systemtype, a user name, and so on. Such parameters may be required toregister the user device 720 in the server 730. The received multimediaidentification signals and parameters are saved in the user device.

For each of the received multimedia identification signals at least onesignature is generated as discussed in detail above. The signatures arethen stored in the database 760. According to one embodiment, thedatabase 760 may further include a “black list” of signatures that areassociated with malicious users or software. In one embodiment, aconcept structure is generated using the signatures generated for thereceived multimedia identification signals, and the concept isassociated with the user device. It should be noted that the user mayfrom time to time send new multimedia identification signals tostrengthen the security of the reliability of the authentication andreduce the number of false negative authentication attempts.

Once the sign-in procedure is completed, the user can access the device720 after a successful completion of an identity verification process.For example, if the user device 720 is a smart phone, the verificationprocess should be performed to unlock the phone. This process may beinitiated by an agent (e.g., a script or an application installed orrunning on the device) or by the operating system of the user device720.

In an embodiment, during the identity verification process, the server730 receives multimedia identification signals corresponding to thesignals provided during the sign-in procedure. For example, if a pictureof the user's face is initially used, then the user is prompted to takea picture of his/her face, and such a picture is sent from the device720 to the server 730. In one embodiment, only a sub-set of multimediaidentification signals of the signals used for the sign up may berequested and used for the verification. The sub-set of multimediaidentification signals may be randomly selected, thereby increasing thebarrier to hack into the user device. For example, if 5 signals are usedto generate the concept during the sign-in process, then only 2 signalsmay be required for verification. Each time the user requests anauthentication, two different signals may be requested.

The server 730 is configured to verify the user identity of the userdevice 720 using the multimedia identification signals received from thedevice 720. With this aim, the received multimedia identificationsignals are processed and analyzed for the purpose of matching thesensory signal to concept structures generated and maintained by the DCCsystem 740. Such a match requires generating at least one signature foreach of the received multimedia identification signals and matching thegenerated signature(s) against the concept structure created during thesign-in procedure and associated with the user device. The signature forthe multimedia identification signals may be generated by means of thesignature generator 750.

In one embodiment, the matching can be performed by providing a query tothe DCC system 740 for finding a match between a concept structurerespective of the user and the received multimedia identificationsignals or their respective signatures. The matching of multimediaidentification signals to a concept structure includes matchingsignatures respective of each signal to the concept structure. Suchmatching results in matching sequences, each sequence is set with athreshold. Crossing of a threshold is considered as a match of theconcept structure, thereby the user identity is verified and the usercan access the device 720.

In another embodiment, signatures saved during the sign-in procedure arematched to signatures generated respective of the multimediaidentification signals received during the verification process. Thematching of the signatures is as discussed above in greater detail. Ifthe signatures are determined to be matched, then the user identity isverified.

The verification of the user identity may allow the user, for example,to access the device 720 (e.g., unlock the device), access informationstored in the device 720, make a payment using the device 720, access anapplication installed on the device 720, and/or for any purpose thatrequires user authentication.

FIG. 8 shows an exemplary and non-limiting flowchart 800 illustratingthe sign-in procedure according to an embodiment. In an embodiment themethod is performed by the server 730. In S810, the operation startswhen a request to establish a new account is received from a user device720. According to one embodiment, S810 may be executed, for example,upon activation of a subscriber identity module (SIM) card on a device720, activation of an agent or application installed on the device 720,or when a connection is established between the server 730 and the userdevice 720 for the first time.

In S820, a user of the user device is prompted to enter one or moremultimedia identification signals. For example, the user is presentedwith a short sentence to repeat in order to avoid fraudulent recordingsof the user's voice. The user may further be prompted to select from aset of pictures stored in the device, or to take a picture ofsurrounding objects. The multimedia identification signals can becaptured by sensors of the device 720.

In S825, the multimedia identification signals captured by the sensorsof the user device 720 are received. In S830, at least one signature isgenerated for each of the received multimedia identification signals.The generation of signatures respective of multimedia content is furtherdiscussed in FIGS. 3 and 4 hereinabove.

In S840, the received multimedia identification signals and theirrespective signatures are stored in a database. In S845, a conceptstructure is created using the signature. The metadata of the conceptstructure may include one or more identifiers of the device 720, forexample, a user name, a device's IP address, a device type, an operatingsystem of the device, and the like. Such details may be entered by auser of the user device 720. The generated concept structure andsignatures are saved in a database for future verification of the useridentification. The database may be the DB 760. In S850, it is checkedwhether additional multimedia identification signals are received, andif so execution continues with S830; otherwise, execution terminates.

As a non-limiting example, upon activation of a new user device, forexample, the user device 720, a request to establish a new account isreceived. A visual parameter, for example a close up image of the user'seyes is captured by a camera of the user's device. In addition, thewords “open sesame,” are presented to the user and an audio input of theuser repeating these words is captured by the microphone of the mobiledevice. One or more signatures are then generated by the signaturegenerator respective of the visual and audio inputs. The signatures arestored in a database accessible by the server 730.

FIG. 9 shows an exemplary and non-limiting flowchart 900 illustrating amethod for verifying a user identity according to one embodiment. In anembodiment the method is performed by the server 730.

In S905, a request to access the user device 720 is received. As notedabove, the request may include unlocking the device 720, accessinginformation stored in the device 720, making a payment using the device720, accessing an application installed on the device 720, and/or forany purpose that requires the user authentication.

In S910, the user is prompted to enter a plurality of multimediaidentification signals corresponding to the multimedia identificationsignals used during the sign-in procedure. In S915, the multimediaidentification signals captured by the sensors of the user device 720are received. As noted above, the multimedia identification signalsprovided in S910 may be only a sub-set of the multimedia identificationsignals previously sent to the server 130. In S915, one or moresignatures are generated respective of the one or more multimediaidentification signals.

In S920, the DCC system (e.g., system 740) is queried to find a matchbetween the concept structure (CS) and the received multimediaidentification signals or their respective signatures. Alternatively, inS920 it is checked whether signatures generated for the receivedmultimedia identification signals match the signatures stored in thedatabase for the user of the device 720. Various embodiments formatching between signatures or between multimedia identification signalsto a concept structure are discussed in detail above.

In S922, it is checked if such a match is found, and if so, executioncontinues with S925; otherwise, execution continues with S935. In S925,a message is sent to the user device 720 that the user is authenticatedto access the device 720.

In S930, it is checked whether additional identity verification requestsare received, and if so execution returns to S910; otherwise, executionterminates. It should be noted that in case the user identity isverified, the multimedia identification signals received at S910 andrelated to the user can be utilized to update the concept structuregenerated respective of the user device in S845, thus allowingidentification of changes in the inputs such as, a haircut, plasticsurgery, voice changes, and so on of the user.

In one embodiment, in order to reduce the number of false negativenotifications, it is checked whether the signatures of the multimediaidentification signals match one of the black-listed signatures storedin the database (S935). If so, execution continues with S940, where amessage is sent to the device to block the access; otherwise, executionreturns to S910 where the user is prompted again to enter new multimediaidentification signals in order to re-perform the verification process.It should be noted that after a predefined number of failed verificationattempts any access to the device 720 is blocked.

According to one embodiment, signatures generated for multimediaidentification signals that did not pass the verification process areadded to the black list of signatures and saved in the database.

The embodiments disclosed herein may be implemented as hardware,firmware, software, or any combination thereof. Moreover, the softwareis preferably implemented as an application program tangibly embodied ona program storage unit or non-transitory computer readable mediumconsisting of parts, or of certain devices and/or a combination ofdevices. The application program may be uploaded to, and executed by, amachine comprising any suitable architecture. Preferably, the machine isimplemented on a computer platform having hardware such as one or morecentral processing units (“CPUs”), a memory, and input/outputinterfaces. The computer platform may also include an operating systemand microinstruction code. The various processes and functions describedherein may be either part of the microinstruction code or part of theapplication program, or any combination thereof, which may be executedby a CPU, whether or not such computer or processor is explicitly shown.In addition, various other peripheral units may be connected to thecomputer platform such as an additional data storage unit and a printingunit. Furthermore, a non-transitory computer readable medium is anycomputer readable medium except for a transitory propagating signal.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the severalembodiments and the concepts contributed by the inventor to furtheringthe art, and are to be construed as being without limitation to suchspecifically recited examples and conditions. Moreover, all statementsherein reciting principles, aspects, and embodiments of the invention,as well as specific examples thereof, are intended to encompass bothstructural and functional equivalents thereof. Additionally, it isintended that such equivalents include both currently known equivalentsas well as equivalents developed in the future, i.e., any elementsdeveloped that perform the same function, regardless of structure.

What is claimed is:
 1. A method for verifying an identity of a useraccessing a user device, comprising: receiving a request to verify theidentity of the user accessing the user device; receiving a current setof multimedia identification signals from the user device in accordancewith the verification request; querying a deep-content-classification(DCC) system to find a match between at least one pre-stored conceptstructure associated with the user to be verified and the current set ofreceived multimedia identification signals, wherein the at least oneconcept structure is created during a sign-in procedure and is based ona grouping of associated clusters, wherein each cluster includes acompressed set of similar signatures generated from an initial set ofpreviously received multimedia identification signals received duringthe sign-in procedure; and sending an authentication notification to theuser upon finding a match between the at least one concept structure andthe current set of received multimedia identification signals.
 2. Themethod of claim 1, further comprising: generating at least one signatureto each of the initial set of multimedia identification signals prior toquerying the DCC system.
 3. The method of claim 1, wherein each of theinitial set of multimedia identification signals is at least one of: animage, a graphic, a video stream, a video clip, an audio stream, anaudio clip, a video frame, a photograph, and images of signals.
 4. Themethod of claim 3, wherein the images of signals are at least one of:medical signals, geophysical signals, subsonic signals, supersonicsignals, electromagnetic signals, infrared signals, an audio signal, avideo signal, coordinates, and a sonography signal.
 5. The method ofclaim 1, wherein each of the current set of multimedia identificationsignals received from the user device is at least one of: captured bythe user device, and stored in the user device.
 6. The method of claim1, wherein the request to verify the user identity accessing the userdevice further includes supplemental information, wherein thesupplemental information comprises at least one of an Internet Protocol(IP) address of the user computing device, a time, a date, and a browsertype of the user computing device.
 7. The method of claim 6, wherein thesupplemental information is used to create metadata from the at leastone concept structure.
 8. The method of claim 7, wherein the sign-inprocedure further comprises: receiving the initial set of multimediaidentification signals from the user device; generating at least onesignature for each multimedia identification signal in the initial setof multimedia identification signals received from the user device;creating the at least one concept structure associated with the userusing the at least one generated signature; associating the metadatawith the created at least one concept structure; and saving the createdat least one concept structure in a database.
 9. The method of claim 7,wherein the current set of received multimedia identification signalscorresponds to the initial set of multimedia identification signals. 10.The method of claim 8, wherein the matching between the at least oneconcept structure and the current set of received multimediaidentification signals further includes: matching signatures in the atleast one concept structure to signatures of the initial set of receivedmultimedia identification signals.
 11. The method of claim 8, whereinthe at least one concept structure is created using the DCC system,wherein the DCC system includes: an attention processor configured togenerate a plurality of items from a multimedia identification signaland to determine which of the generated items are of interest forsignature generation; a signature generator configured to generate atleast one signature responsive to at least one item of interest; and aconcept generator configured to match between the at least one signaturegenerated responsive to at least one of the items of interest and a setof signature reduced clusters associated with a plurality of clusterstructures.
 12. A non-transitory computer readable medium having storedthereon instructions for causing one or more processing units to executethe method according to claim
 1. 13. The method of claim 1, wherein eachsignature is generated by a signature generator system, wherein thesignature generator system includes a plurality of computational coresconfigured to receive a plurality of unstructured data elements, eachcomputational core of the plurality of computational cores havingproperties that are at least partly statistically independent of otherof the computational cores, the properties are set independently of eachother core.
 14. A server for verifying an identity of a user accessing auser device, comprising: an interface to a network for receiving arequest to verify the identity of the user accessing the user device andfor receiving a current set of multimedia identification signals fromthe user device; a processor; and a memory connected to the processor,the memory contains instructions that, when executed by the processor,configure the server to: query a deep-content-classification (DCC)system to find a match between at least one concept structure associatedwith the user and a current set of multimedia identification signals,wherein the concept structure is created during a sign-in procedure andis based on a grouping of associated clusters, where each clusterincludes a compressed set of similar signatures generated from aninitial set of multimedia identification signals received during thesign-in procedure; and send an authentication notification upon findinga match between the at least one concept structure and the current setof received multimedia identification signals.
 15. The server of claim14, wherein the server is further configured to: generate at least onesignature to each of the initial set of multimedia identificationsignals prior to querying of the DCC system.
 16. The server of claim 14,wherein each of the initial set of multimedia identification signals isat least one of: an image, a graphic, a video stream, a video clip, anaudio stream, an audio clip, a video frame, a photograph, and images ofsignals.
 17. The server of claim 14, wherein the initial set ofmultimedia identification signals is at least one of: captured by theuser device, and stored in the user device.
 18. The server of claim 14,wherein the request further includes supplemental information, whereinthe supplemental information comprises at least one of an Internetprotocol (IP) address of the user computing device, a time, a date, anda browser type of the IP address of the user computing device, whereinthe supplemental information is used to create a metadata of the conceptstructure.
 19. The server of claim 18, wherein the server is furtherconfigured to: receive an initial set of multimedia identificationsignals from the user device; generate at least one signature for eachsignal in the initial set of multimedia identification signals; createthe at least one concept structure associated with the user based on thegenerated signatures; associate the metadata with the created at leastone concept structure; and save the created concept structure in adatabase.
 20. The server of claim 19, wherein the current set ofreceived multimedia identification signals correspond to the initial setof multimedia identification signals, wherein further the current set ofreceived multimedia identification signals is a subset of the initialset of multimedia identification signals.
 21. The server of claim 19,wherein the DCC system is further configured to match signatures in theat least one concept structure to signatures of the received set ofcurrent multimedia identification signals.
 22. The server of claim 19,wherein the at least one concept structure is created using the DCCsystem, wherein the DCC system includes: an attention processorconfigured to generate a current set of items from a multimediaidentification signal and to determine which of the generated items areof interest for signature generation; a signature generator configuredto generate at least one signature responsive to at least one item ofinterest; and a concept generator configured to match between the atleast one signature generated responsive to at least one of the items ofinterest and a current set of signature reduced clusters associated witha current set of cluster structures.
 23. The server of claim 14, whereinthe DCC system is integrated in the server.